Natural Language Processing for Improved Characterization of COVID-19 Symptoms: An Observational Study of 350,000 Patients in a Large Integrated Healthcare System.

BACKGROUND: Natural language processing (NLP) of unstructured text from Electronic Medical Records (EMR) can improve characterization of COVID-19 signs and symptoms, but large-scale studies demonstrating the real-world application and validation of NLP for this purpose are limited. OBJECTIVE: To assess the contribution of NLP when identifying COVID-19 signs and symptoms from EMR. METHODS: This study was conducted in Kaiser Permanente Southern California, a large integrated healthcare system using data from all patients with positive SARS-CoV-2 laboratory tests from March 2020 to May 2021. An NLP algorithm was developed to extract free text from EMR on 12 established signs and symptoms of COVID-19, including fever, cough, headache, fatigue, dyspnea, chills, sore throat, myalgia, anosmia, diarrhea, vomiting/nausea and abdominal pain. The proportion of patients reporting each symptom and the corresponding onset dates were described before and after supplementing structured EMR data with NLP-extracted signs and symptoms. A random sample of 100 chart-reviewed and adjudicated SARS-CoV-2 positive cases were used to validate the algorithm performance. RESULTS: A total of 359,938 patients (mean age: 40.4 years; 53% female) with confirmed SARS-CoV-2 infection were identified over the study period. The most common signs and symptoms identified through NLP-supplemented analyses were cough (61%), fever (52%), myalgia (43%), and headache (40%). The NLP algorithm identified an additional 55,568 (15%) symptomatic cases that were previously defined as asymptomatic using structured data alone. The proportion of additional cases with each selected symptom identified in NLP-supplemented analysis varied across the selected symptoms, from 29% of all records for cough, to 61% of all records with nausea or vomiting. Of 295,305 symptomatic patients, the median time from symptom onset to testing was 3 days using structured data alone, whereas the NLP-algorithm identified signs or symptoms approximately one day earlier. When validated against chart-reviewed cases, the NLP algorithm successfully identified most signs and symptoms with consistently high sensitivity (ranging from 87% to 100%) and specificity (94% to 100%). CONCLUSIONS: These findings demonstrate that NLP can identify and characterize a broad set of COVID-19 signs and symptoms from unstructured data within the EMR, with enhanced detail and timeliness compared with structured data alone.

Retrospective cohort study to assess the association between treatment with tocilizumab and mortality among mechanically ventilated patients with COVID-19.

OBJECTIVES: Assess the association between tocilizumab administration and clinical outcomes among mechanically ventilated patients with COVID-19 pneumonia. DESIGN: Retrospective cohort study. SETTING: Large integrated health system with 9 million members in California, USA. PARTICIPANTS: 4185 Kaiser Permanente members hospitalised with COVID-19 pneumonia requiring invasive mechanical ventilation (IMV). INTERVENTIONS: Receipt of tocilizumab within 10 days of initiation of IMV. OUTCOME MEASURES: Using a retrospective cohort of consecutive patients hospitalised with COVID-19 pneumonia who required IMV in a large integrated health system in California, USA, we assessed the association between tocilizumab administration and 28-day mortality, time to extubation from IMV and time to hospital discharge. RESULTS: Among 4185 patients, 184 received tocilizumab and 4001 patients did not receive tocilizumab within 10 days of initiation of IMV. After inverse probability weighting, baseline characteristics were well balanced between groups. Patients treated with tocilizumab had a similar risk of death in the 28 days after intubation compared with patients not treated with tocilizumab (adjusted HR (aHR), 1.21, 95% CI 0.98 to 1.50), but did have a significantly longer time-to-extubation (aHR 0.71; 95% CI 0.57 to 0.88) and time-to-hospital-discharge (aHR 0.66; 95% CI 0.50 to 0.88). However, patients treated with tocilizumab ≤2 days after initiation of IMV had a similar risk of mortality (aHR 1.47; 95% CI 0.96 to 2.26), but significantly shorter time-to-extubation (aHR 0.37; 95% CI 0.23 to 0.58) and time-to-hospital-discharge (aHR 0.31; 95% CI CI 0.17 to 0.56) compared with patients treated with tocilizumab 3-10 days after initiation of IMV. CONCLUSIONS: Among mechanically ventilated patients with COVID-19, the risk of death in the 28-day follow-up period was similar, but time-to-extubation and time-to-hospital-discharge were longer in patients who received tocilizumab within 10 days of initiation of IMV compared with patients who did not receive tocilizumab.

Severity of COVID-19 and adverse long-term outcomes: a retrospective cohort study based on a US electronic health record database.

OBJECTIVE: To identify potential risk factors for adverse long-term outcomes (LTOs) associated with COVID-19, using a large electronic health record (EHR) database. DESIGN: Retrospective cohort study. Patients with COVID-19 were assigned into subcohorts according to most intensive treatment setting experienced. Newly diagnosed conditions were classified as respiratory, cardiovascular or mental health LTOs at >30-≤90 or >90-≤180 days after COVID-19 diagnosis or hospital discharge. Multivariate regression analysis was performed to identify any association of treatment setting (as a proxy for disease severity) with LTO incidence. SETTING: Optum deidentified COVID-19 EHR dataset drawn from hospitals and clinics across the USA. PARTICIPANTS: Individuals diagnosed with COVID-19 (N=57 748) from 20 February to 4 July 2020. MAIN OUTCOMES: Incidence of new clinical conditions after COVID-19 diagnosis or hospital discharge and the association of treatment setting (as a proxy for disease severity) with their risk of occurrence. RESULTS: Patients were assigned into one of six subcohorts: outpatient (n=22 788), emergency room (ER) with same-day COVID-19 diagnosis (n=11 633), ER with COVID-19 diagnosis≤21 days before ER visit (n=2877), hospitalisation without intensive care unit (ICU; n=16 653), ICU without ventilation (n=1837) and ICU with ventilation (n=1960). Respiratory LTOs were more common than cardiovascular or mental health LTOs across subcohorts and LTO incidence was higher in hospitalised versus non-hospitalised subcohorts. Patients with the most severe disease were at increased risk of respiratory (risk ratio (RR) 1.86, 95% CI 1.56 to 2.21), cardiovascular (RR 2.65, 95% CI 1.49 to 4.43) and mental health outcomes (RR 1.52, 95% CI 1.20 to 1.91) up to 6 months after hospital discharge compared with outpatients. CONCLUSIONS: Patients with severe COVID-19 had increased risk of new clinical conditions up to 6 months after hospital discharge. The extent that treatment setting (eg, ICU) contributed to these conditions is unknown, but strategies to prevent COVID-19 progression may nonetheless minimise their occurrence.

Prognostic model to identify and quantify risk factors for mortality among hospitalised patients with COVID-19 in the USA.

OBJECTIVES: To develop a prognostic model to identify and quantify risk factors for mortality among patients admitted to the hospital with COVID-19. DESIGN: Retrospective cohort study. Patients were randomly assigned to either training (80%) or test (20%) sets. The training set was used to fit a multivariable logistic regression. Predictors were ranked using variable importance metrics. Models were assessed by C-indices, Brier scores and calibration plots in the test set. SETTING: Optum de-identified COVID-19 Electronic Health Record dataset including over 700 hospitals and 7000 clinics in the USA. PARTICIPANTS: 17 086 patients hospitalised with COVID-19 between 20 February 2020 and 5 June 2020. MAIN OUTCOME MEASURE: All-cause mortality while hospitalised. RESULTS: The full model that included information on demographics, comorbidities, laboratory results, and vital signs had good discrimination (C-index=0.87) and was well calibrated, with some overpredictions for the most at-risk patients. Results were similar on the training and test sets, suggesting that there was little overfitting. Age was the most important risk factor. The performance of models that included all demographics and comorbidities (C-index=0.79) was only slightly better than a model that only included age (C-index=0.76). Across the study period, predicted mortality was 1.3% for patients aged 18 years old, 8.9% for 55 years old and 28.7% for 85 years old. Predicted mortality across all ages declined over the study period from 22.4% by March to 14.0% by May. CONCLUSION: Age was the most important predictor of all-cause mortality, although vital signs and laboratory results added considerable prognostic information, with oxygen saturation, temperature, respiratory rate, lactate dehydrogenase and white cell count being among the most important predictors. Demographic and comorbidity factors did not improve model performance appreciably. The full model had good discrimination and was reasonably well calibrated, suggesting that it may be useful for assessment of prognosis.